Discriminative Feature Selection Research Articles

Effective Data Reduction Using Discriminative Feature Selection Based on Principal Component Analysis

Effective data reduction must retain the greatest possible amount of informative content of the data under examination. Feature selection is the default for dimensionality reduction, as the relevant features of a dataset are usually retained through this method. In this study, we used unsupervised learning to discover the top-k discriminative features present in the large multivariate IoT dataset used. We used the statistics of principal component analysis to filter the relevant features based on the ranks of the features along the principal directions while also considering the coefficients of the components. The selected number of principal components was used to decide the number of features to be selected in the SVD process. A number of experiments were conducted using different benchmark datasets, and the effectiveness of the proposed method was evaluated based on the reconstruction error. The potency of the results was verified by subjecting the algorithm to a large IoT dataset, and we compared the performance based on accuracy and reconstruction error to the results of the benchmark datasets. The performance evaluation showed consistency with the results obtained with the benchmark datasets, which were of high accuracy and low reconstruction error.

Boosted unsupervised feature selection for tumor gene expression profiles

AbstractIn an unsupervised scenario, it is challenging but essential to eliminate noise and redundant features for tumour gene expression profiles. However, the current unsupervised feature selection methods treat all samples equally, which tend to learn discriminative features from simple samples. Besides, it is difficult for these methods to guide the feature selection process by multiple feature subspaces comprehensively. In order to address these problems, a boosted unsupervised feature selection method (BoostUFS) is proposed for tumour gene expression profiles. Specifically, the authors design a boosting scheme to sequentially learn multiple compressed feature subspaces by focusing on ambiguous samples. The uncertainty of samples and the confidence of feature subspaces can be evaluated adaptively by minimising the overall loss of feature subspaces learning. Furthermore, we provide a consensus objective function with L2,1‐norm regularisation to combine these weighted feature subspaces and select discriminative features. Extensive experiments on several real‐world datasets of tumour gene expression profiles are carried out to demonstrate the effectiveness of the proposed method.

Fault Diagnosis of a Multistage Centrifugal Pump Using Explanatory Ratio Linear Discriminant Analysis.

This study introduces an innovative approach for fault diagnosis of a multistage centrifugal pump (MCP) using explanatory ratio (ER) linear discriminant analysis (LDA). Initially, the method addresses the challenge of background noise and interference in vibration signals by identifying a fault-sensitive frequency band (FSFB). From the FSFB, raw hybrid statistical features are extracted in time, frequency, and time-frequency domains, forming a comprehensive feature pool. Recognizing that not all features adequately represent MCP conditions and can reduce classification accuracy, we propose a novel ER-LDA method. ER-LDA evaluates feature importance by calculating the explanatory ratio between interclass distance and intraclass scatteredness, facilitating the selection of discriminative features through LDA. This fusion of ER-based feature assessment and LDA yields the novel ER-LDA technique. The resulting selective feature set is then passed into a k-nearest neighbor (K-NN) algorithm for condition classification, distinguishing between normal, mechanical seal hole, mechanical seal scratch, and impeller defect states of the MCP. The proposed technique surpasses current cutting-edge techniques in fault classification.

Self-paced semi-supervised feature selection with application to multi-modal Alzheimer’s disease classification

Semi-supervised multi-modal learning has attracted much attention due to the expense and scarcity of data labels, especially in disease diagnosis field. Most existing methods follow the paradigm by iteratively inferring the pseudo-labels of unlabeled data and add them into training sequence, but they ignore the reliability of those pseudo-labels, where inaccurate and wrong supervision will lead to negative influence on model learning. In this paper, we propose a Self-paced Semi-supervised Multi-modal Feature Selection (SSMFS) method, and apply it to Alzheimer’s disease classification. Specifically, SSMFS projects multi-modal biomedical data into the common label space with discriminative feature selection. Under the guidance of prior multi-modal similarity graphs, a unified graph is adaptively learned and embedded to preserve the neighborhood structures. More importantly, SSMFS dynamically investigates the discriminability and credibility of pseudo-labels, and adaptively assigns a weight to each unlabeled sample via self-paced learning such that the negative influence of wrong supervision can be reduced. Finally, a multi-kernel support vector machine is used to fuse the selected multi-modal features for final disease prediction. Experimental results on the Alzheimer’s Disease Neuroimaging Initiative (ADNI) dataset demonstrate the effectiveness of our method.

A Comparative Analysis of Morphological Characteristics between Endangered Local Prickly Pear and the Newly Introduced Dactylopius opuntiae-Resistant Species in Eastern Morocco.

Prickly pear serves as a significant source of income for farmers worldwide, with production taking place in temperate, subtropical, and cold regions. The objective of the present investigation is to explore the morphological parameters of Opuntia robusta and Opuntia dillenii which are resistant to the white cochineal (Dactylopius opuntiae), as well as the local prickly pear that is currently threatened with extinction. This investigation aims to evaluate the feasibility of replacing the endangered local prickly pear with the recently introduced species O. robusta and O. dillenii. This analysis is based on a comprehensive assessment of 26 qualitative and 25 quantitative traits pertaining to cladodes and fruits. In terms of species differentiation and the selection of discriminative features, this study demonstrates the effectiveness of various statistical methods, as well as the analysis carried out according to the descriptors recommended by the International Union for the Protection of New Varieties of Plants (UPOV). Of the 51 parameters evaluated, 13 qualitative and 23 quantitative characters are significant in differentiating the species under study. This underscores the importance of quantitative traits in distinguishing different prickly pear species. Furthermore, color is identified as a crucial characteristic for discriminating between the studied samples. O. robusta is characterized by its high fruit weight, large size, greater pulp content, and high pulp-to-peel ratio, all of which are desirable traits for fresh consumption. Additionally, O. robusta has the highest number of fully developed seeds, making it an attractive option for use in the cosmetic industry. This characteristic renders the O. robusta a potential substitute for the endangered ecotype . However, O. robusta is distinguished by its short stalk, which poses a challenge for fruit harvesting and leaves it susceptible to physical damage and quality loss. Conversely, O. dillenii displays a low pulp content, which serves as a critical indicator of fruit quality. The only desirable agronomic trait of this species is its elevated seed content, which has the potential for utilization in oil production for the cosmetic industry.

Local sparse discriminative feature selection

Feature selection has been widely used in machine learning for a long time. In this paper, we propose a supervised local sparse discriminative feature selection method named LSDFS to obtain sparse features by imposing ℓ2,0-norm constraint on transformation matrix. Differently from traditional approaches, our method does not require approximation or relaxation schemes, such as ℓ2,p-norm to solve long-standing challenge. Our method is based on the trace difference form of Linear Discriminant Analysis (LDA), which can efficiently obtain discriminative information in low-dimensional space. In order to explore the local structure of data which contains more discriminative information, we adopt a sparse connections graph between anchor points and data points instead of fully-connected graph with time-consuming, and add a decay parameter to avoid trivial solutions, making the model more precisely. Extensive experiments conducted on synthetic datasets and several real-world datasets have demonstrated the advantages of our method.

Soft Label Guided Unsupervised Discriminative Sparse Subspace Feature Selection

Soft Label Guided Unsupervised Discriminative Sparse Subspace Feature Selection

Discriminative feature selection using signed Laplacian restricted Boltzmann machine for speed and generalization improvement of high dimensional data classification

Deep classifiers require lots of computations due to the use of multi-layer structure and processing of bulk input samples. Selecting effective features that have the same accuracy as full features, has recently been considered. Using feature selection, less calculation is needed to update the weights and consequently the speed of the training/testing process is increased. In this research, a new deep feature selection method called Discriminative Deep Feature Selection using Signed Laplacian Restricted Boltzmann Machine (DDFS-SLRBM) is proposed. In the approach, full training samples are fed into a SLRBM model. In the updates, the weight matrix is recomputed using the neighborhood matrix of similar and dissimilar classes which results in the discriminative property of the approach. Then, selected features are identified based on the minimum reconstruction error criterion. The efficiency of the proposed algorithm is demonstrated by performing different experiments on MNIST, GISETTE and Protein datasets. The experiments show that the proposed method is able to improve the classification accuracy and the generalizability of the approach while reducing the processing time. Also, the proposed approach shows scalability on the type of the problem, number of original features, number of samples and the number of final selected features.

Unsupervised Discriminative Feature Selection via Contrastive Graph Learning.

Due to many unmarked data, there has been tremendous interest in developing unsupervised feature selection methods, among which graph-guided feature selection is one of the most representative techniques. However, the existing feature selection methods have the following limitations: (1) All of them only remove redundant features shared by all classes and neglect the class-specific properties; thus, the selected features cannot well characterize the discriminative structure of the data. (2) The existing methods only consider the relationship between the data and the corresponding neighbor points by Euclidean distance while neglecting the differences with other samples. Thus, existing methods cannot encode discriminative information well. (3) They adaptively learn the graph in the original or embedding space. Thus, the learned graph cannot characterize the data's cluster structure. To solve these limitations, we present a novel unsupervised discriminative feature selection via contrastive graph learning, which integrates feature selection and graph learning into a uniform framework. Specifically, our model adaptively learns the affinity matrix, which helps characterize the data's intrinsic and cluster structures in the original space and the contrastive learning. We minimize l1,2 -norm regularization on the projection matrix to preserve class-specific features and remove redundant features shared by all classes. Thus, the selected features encode discriminative information well and characterize the discriminative structure of the data. Generous experiments indicate that our proposed model has state-of-the-art performance.

Manifold embedded global and local discriminative features selection for single-shot multi-categories clothing recognition and retrieval

PurposeSingle-shot multi-category clothing recognition and retrieval play a crucial role in online searching and offline settlement scenarios. Existing clothing recognition methods based on RGBD clothing images often suffer from high-dimensional feature representations, leading to compromised performance and efficiency.Design/methodology/approachTo address this issue, this paper proposes a novel method called Manifold Embedded Discriminative Feature Selection (MEDFS) to select global and local features, thereby reducing the dimensionality of the feature representation and improving performance. Specifically, by combining three global features and three local features, a low-dimensional embedding is constructed to capture the correlations between features and categories. The MEDFS method designs an optimization framework utilizing manifold mapping and sparse regularization to achieve feature selection. The optimization objective is solved using an alternating iterative strategy, ensuring convergence.FindingsEmpirical studies conducted on a publicly available RGBD clothing image dataset demonstrate that the proposed MEDFS method achieves highly competitive clothing classification performance while maintaining efficiency in clothing recognition and retrieval.Originality/valueThis paper introduces a novel approach for multi-category clothing recognition and retrieval, incorporating the selection of global and local features. The proposed method holds potential for practical applications in real-world clothing scenarios.

Multi-label feature selection via adaptive dual-graph optimization

As in single-label learning, multi-label learning (MLL) also suffers from the problem of “the curse of dimensionality” due to the redundancy of features in the original data. To address this problem, extensive research has been conducted to explore the various feature selection algorithms aiming to eliminate irrelevant and redundant features. Among these, the manifold learning based methods have gained significant attention for their ability to capture local label correlations. However, previous approaches have mainly focused on extracting information from data space, overlooking valuable information in feature space. In addition, these methods conduct manifold learning and feature selection independently, thereby degrading the learning performance. To tackle this problem, we propose a novel multi-label feature selection method based on adaptive dual-graph optimization (MFS-ADGO) to learn the geometric structure of the data and select discriminative feature subsets simultaneously. Specifically, a dual-graph manifold regularization is constructed first, to locally capture label correlations in the data and feature spaces for discriminative features selection. A constraint is then imposed on the similarity matrices to adaptively learn the local geometric structure of the data during the feature selection, thereby capturing more reliable information from the original data. To facilitate feature selection, an objective function incorporating the l2,1−2-norm is proposed, along with an iterative updating algorithm. Extensive experiments unequivocally demonstrated the efficacy of MFS-ADGO in enhancing state-of-the-art feature selection methods in the realm of MLL.

Comparing univariate filtration preceding and succeeding PLS-DA analysis on the differential variables/metabolites identified from untargeted LC-MS metabolomics data

BackgroundPLS-DA of high-dimensional metabolomics data is frequently employed to capture the most pertinent features to sample classification. But the presence of numerous insignificant input features could distort the PLS-DA model, blow up and scramble the selected differential features. Usually, univariate filtration is subsequently complemented to refine the selected features, but often giving unstable results. Whereas by precluding insignificant features through univariate data prefiltration assessed by FDR adjusted p-value, PLS-DA can generate more stable and reliable differential features. We explored and compared these two data analysis procedures to gain insights into the underlying mechanisms responsible for the disparate results. ResultsThe effect of univariate data filtration preceding and succeeding PLS-DA analysis on the identified discriminative features/metabolites was investigated using LC-MS data acquired on the samples of human serum and C. elegans extracts, with and without metabolite standards spiked to simulate the treated and control groups of biological samples. It was shown that the univariate data prefiltration before PLS-DA usually gave less but more stable and likely more reliable and meaningful differential features, while PLS-DA applied directly to the original data could be affected by the presence of insignificant features and orthogonal noise. Large number of insignificant variables and orthogonal noise could distort the generated PLS-DA model and affect the p(corr) value, and artificially inflate the calculated VIP values of relevant features due to the increased total number of input features for model construction, thus leading to more false positives selected by the conventional VIP threshold of 1.0. Significance and noveltyUnivariate data filtration preceding PLS-DA was important for the identification of reliable differential features if using a conventional threshold of VIP of 1.0. Presence of insignificant features could distort the PLS-DA model and inflate VIP values. Appropriate VIP threshold is associated with the numbers of input features and the model components. For PLS-DA without univariate prefiltration, threshold of VIP larger than 1.0 is recommended for the selection of discriminative features to reduce the false positives.

Discriminative multi-label feature selection with adaptive graph diffusion

Feature selection can alleviate the problem of the curse of dimensionality by selecting more discriminative features, which plays an important role in multi-label learning. Recently, embedded feature selection methods have received increasing attentions. However, most existing methods learn the low-dimensional embeddings under the guidance of the local structure between the original instance pairs, thereby ignoring the high-order structure between instances and being sensitive to noise in the original features. To address these issues, we propose a feature selection method named discriminative multi-label feature selection with adaptive graph diffusion (MFS-AGD). Specifically, we first construct a graph embedding learning framework equipped with adaptive graph diffusion to uncover a latent subspace that preserves the higher-order structure information between four tuples. Then, the Hilbert–Schmidt independence criterion (HSIC) is incorporated into the embedding learning framework to ensure the maximum dependency between the latent representation and labels. Benefiting from the interactive optimization of the feature selection matrix, latent representation and similarity graph, the selected features can accurately explore the higher-order structural and supervised information of data. By further considering the correlation between labels, MFS-AG is extended to a more discriminative version,i.e., LMFS-AG. Extensive experimental results on various benchmark data sets validate the advantages of the proposed MFS-AGD and LMFS-AGD methods.

A novel scheme based on modified hierarchical time-shift multi-scale amplitude-aware permutation entropy for rolling bearing condition assessment and fault recognition

To avoid production interruptions and equipment damage caused by rolling bearing failure, this study presents a novel diagnosis scheme applicable to both condition monitoring and fault recognition. In detail, to fully exploit feature information, a modified Hierarchical Time-Shift Multi-scale Amplitude-Aware Permutation Entropy (MHTSMAAPE) is put forward to map the raw vibration signal into a high-dimensional vector space of features. This method is developed from the AAPE algorithm and integrates the time-shift procedure and modified hierarchical analysis, extracting more feature details while ensuring entropy stability. Afterwards, the averaged value of the feature vector is regarded as an indicator to assess the condition of rolling bearing, this indicator exceeds a specific value indicates the rolling bearing steps into faulty. Then, the Unsupervised Discriminative Feature Selection (UDFS) algorithm is first introduced to directly pick fault-sensitive features from the vector space, relieving the calculation burden of the fault recognition task. The fault identification algorithm is based on Harris Hawks Optimization-Joint Opposite Selection optimized for Support Vector Machine (HHO-JOS-SVM) to improve classification accuracy. Finally, the proposed scheme fused by MHTSMAAPE, UDFS, and HHO-JOS-SVM is validated in two different cases with four common evaluation metrics, respectively. The experimental results demonstrate its reliability and robustness in rolling bearing condition assessment and fault recognition.

A linear discriminant analysis-based algorithm for identifying anomalous traffic in large-scale networks

Abstract To protect network security, this paper develops a large-scale network anomalous traffic identification algorithm that utilizes the linear discriminant analysis method to intercept network anomalous traffic. Firstly, the classification of large-scale network anomalous traffic is explored, and the SSAE algorithm is combined with the feature selection of large-scale network traffic on the basis of network flow feature extraction. Secondly, data dimensionality reduction of network anomalous traffic using linear discriminant analysis and feature selection of large-scale network traffic based on SSAE to identify network anomalous traffic. Finally, the CICIDS2017 dataset and the NSL-KDD dataset are used to experimentally analyze the effect and performance of feature selection and anomaly identification algorithms. The results show that the classification accuracy of the feature selection algorithm is 0.989, the 10-dimensional optimal features selected are (12,6,5,38,29,3,33,35,36,40), and the recognition result is 0.803 for normal network traffic and 0.197 for anomalous traffic, with an overall recognition error of 0.003, and a performance of more than 0.988.

Improvement of motor imagery electroencephalogram decoding by iterative weighted Sparse-Group Lasso

Discriminative feature selection is vital for enhancing motor imagery decoding performance in electroencephalogram (EEG) signals. However, existing feature optimization methods have not sufficiently explored the intrinsic attribute distribution of features and their associations with the target class, which could result in spurious correlations between optimized features and class labels, yielding suboptimal performance. Therefore, this study proposed an iterative Weighted Sparse-Group Lasso (iWSGL) model for optimizing Common Spatial Pattern (CSP)-based high-dimensional features, thus further enhancing the decoding accuracy of motor imagery. Specifically, the affinity propagation (AP) clustering algorithm was utilized to adaptively partition the high-dimensional features into multiple groups based on the underlying relationships among them. To evaluate the significance of individual feature within each group and the overall significance of the groups themselves, a weight calculation method was proposed based on conditional entropy. With the weights and feature structural information, a weighted sparse regression model was devised within the iterative Sparse-Group Lasso (iSGL) framework to jointly optimize the CSP-based high-dimensional features. The performance of the proposed method was validated on three datasets using the support vector machine (SVM). The experimental results exhibited the exceptional superiority of the proposed method over the current CSP and its variants, demonstrating its remarkable performance. These findings imply that the proposed model can offer a novel optimization strategy for enhancing pattern recognition of brain intentions in Brain-Computer Interface (BCI) applications.

Development of tuned hybrid fuzzy and BiLSTM-based epileptic seizure classification model with stacked 1D-CNN-fisher discriminant feature selection

ABSTRACT Electroencephalogram (EEG) is a signal which consists of different sinusoidal components with a dense frequency spectrum. The existing methods are time-consumingand includes inconsistencies in judgment among seizure classification. Therefore, it is very difficult to accurately detect epileptic seizures from the recorded EEG. Therefore, a new epileptic seizure detection model is developed with hybrid deep-structured technology. The EEG signal data obtained from the standard online resources which are given into the pre-processing phase with the help of band pass filtering and smoothing techniques. Then, the 5-level Discrete Wavelet Transform (5-level DWT) is used for signal decomposition to get the decomposed signals. 1-D stacked Convolutional Neural Network (1-D stacked CNN) is utilized for the extraction of features. After, the feature selection process is executed by utilizing the Fisher Discriminant Analysis (FDA). The selected features are subjected to the classification phase by Tuned Hybrid Fuzzy Bi-directional Long Short-Term Memory (THFBi-LSTM). Here, the parameter optimization takes place with hybridized optimization algorithm of Probability-based Dingo Coyote Optimization (P-DCO). In the overall simulation estimation, the offered approach achieves a 97% accuracy rate and also a 92% F1-score rate. Thus, the experimental results are reveled that it is better than the other baseline approaches.

HONMF: integration analysis of multi-omics microbiome data via matrix factorization and hypergraph.

The accumulation of multi-omics microbiome data provides an unprecedented opportunity to understand the diversity of bacterial, fungal, and viral components from different conditions. The changes in the composition of viruses, bacteria, and fungi communities have been associated with environments and critical illness. However, identifying and dissecting the heterogeneity of microbial samples and cross-kingdom interactions remains challenging. We propose HONMF for the integrative analysis of multi-modal microbiome data, including bacterial, fungal, and viral composition profiles. HONMF enables identification of microbial samples and data visualization, and also facilitates downstream analysis, including feature selection and cross-kingdom association analysis between species. HONMF is an unsupervised method based on hypergraph induced orthogonal non-negative matrix factorization, where it assumes that latent variables are specific for each composition profile and integrates the distinct sets of latent variables through graph fusion strategy, which better tackles the distinct characteristics in bacterial, fungal, and viral microbiome. We implemented HONMF on several multi-omics microbiome datasets from different environments and tissues. The experimental results demonstrate the superior performance of HONMF in data visualization and clustering. HONMF also provides rich biological insights by implementing discriminative microbial feature selection and bacterium-fungus-virus association analysis, which improves our understanding of ecological interactions and microbial pathogenesis. The software and datasets are available at https://github.com/chonghua-1983/HONMF.

Structural regularization based discriminative multi-view unsupervised feature selection

Multi-view unsupervised feature selection (MUFS) has recently aroused considerable attention, which can select the compact representative feature subset from original multi-view data. Despite the promising preliminary performance, most previous MUFS methods fail to explore the discriminative ability of multi-view data. In addition, they usually utilize spectral analysis to maintain the geometrical structure, which will inevitably increase the difficulty of parameter selection. To address these issues, we present a novel MUFS method, named structural regularization based discriminative multi-view unsupervised feature selection (SDFS). Specifically, we calculate the similarity matrix of sample space from different views and automatically weight each view-specific graph to learn a consensus similarity graph, in which these two types of graphs can promote each other. Further, we treat the learned latent representation as the cluster indicator, and employ a graph regularization without introducing additional parameters to maintain the geometrical structure of data. Besides, a simple yet efficient iterative updating algorithm with theoretical convergence property is developed. Extensive experiments on several benchmark datasets verify that the designed model is superior to several state-of-the-art MUFS models.

Discriminative feature learning and selection with label-induced sparse filtering for intelligent fault diagnosis of rotating machinery

Representation learning has demonstrated its powerful potential in intelligent fault diagnosis of rotating machinery, where sparse filtering (SF) is a popular and promising representation learning method. However, the currently SF-based approaches still have two limitations: (1) discriminant ability of the learned features is insufficient when handling complex signals collected from variable operating conditions and (2) they are incapable of selecting the most relevant and important features for fault classification. To address these issues, label-induced sparse filtering (LISF) is proposed by introducing a fully-connected label layer. On the one hand, discriminant information encoded in labels is exploited in LISF’s training, so discriminative ability of the learned features can be significantly enhanced. On the other hand, the importance of each feature is measured by a projection matrix, so the top-ranked relevant features can be filtered accordingly. Our method is evaluated by experiments on two case studies. It shows that LISF is able to learn discriminative features from complex signals, and achieves excellent diagnosis results over the existing advanced RL methods. Moreover, LISF can automatically select a set of a specified number of key features for fault classification, so as to improve diagnosis efficiency greatly.